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NIGHT  VISION  SYSTEMS  DEVELOPMENTAL  FLIGHT 
TESTING  AND  EVALUATION 

Background 

Night  vision  systems  (NVSs)  have  revolutionized  the  combat  operations  of  war 
fighters  by  denying  the  enemy  the  cover  of  darkness,  and  by  providing  greatly 
enhanced  night  flight  capabilities.  Navigation,  safety,  loadmaster  coordination  of  air 
drop,  target  detection  and  target  acquisition  have  all  been  dramatically  improved  due 
to  the  use  and  advances  in  NVSs.  Once  limited  to  weapon  aiming  and  surveillance, 
modem  NVSs  provide  aircrew’s  with  enhanced  situation  awareness  of  the  night  flight 
envelope,  as  well  as  a  more  effective  way  to  complete  military  objectives.  As  NVSs 
become  more  common,  and  more  capable,  the  role  of  the  United  States  Air  Force 
Flight  Test  Center  (AFFTC)  in  testing  those  systems  on  aircraft  continues  to  expand. 

Introduction 

One  of  the  roles  of  the  AFFTC  is  to  test  and  evaluate  NVSs  in  their  developmental 
stages.  Typically,  NVSs  are  received  from  the  manufacturer  after  laboratory  testing 
has  been  completed;  however,  these  systems  are  often  not  production  representative. 
The  role  of  the  AFFTC  is  to  test  and  evaluate  these  developmental  systems  so  that 
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any  problems  are  corrected  before  they  enter  the  operational  test  stage  where  they 
must  be  fully  production  representative.  A  separate  United  States  Air  Force  (USAF) 
organization  called  the  Air  Force  Operational  Test  and  Evaluation  Center,  or 
AFOTEC,  becomes  involved  at  that  stage  of  product  development.  Generally,  the 
tests  completed  at  the  AFFTC  involve  dynamic  and  integrated  flight  test  conditions 
not  possible  in  a  laboratory.  The  role  of  the  flight  test  center  is  to  determine  how 
successfully  the  NVSs  will  integrate  with  the  aircraft  in  which  they  will  be  used. 
System  and  subsystem  compatibility,  utility,  reliability,  and  effectiveness  are 
common  areas  of  concern,  and  these  areas  must  be  tested  before  the  system  is 
released  to  operational  testing  or  subsequent  operational  use. 

Often  a  NVS  will  be  designed  for  use  in  multiple  aircraft  and  the  integration  of  that  NVS 
with  all  the  subsystems  of  a  variety  of  aircraft  becomes  a  further  goal  of  flight  test. 

While  not  truly  operational  in  character,  the  testing  employed  at  the  AFFTC  attempts 
to  mirror  operational  conditions  in  some  critical  areas,  and  one  of  those  approaches  is 
called  envelope  expansion  testing.  During  this  testing,  the  system  under  test  may  be 
subjected  to  extremes  not  even  encountered  in  operational  use.  In  other  words, 
determining  a  system’s  operating  limits  must  be  done  before  any  operational  military 
use  can  occur.  This  type  of  testing  is  always  done  under  highly  controlled  test 
conditions  using  trained  test  pilots. 

The  advantages  to  AFFTC  developmental  testing,  are  three  fold:  1)  structured  test 
points  allow  variables  of  interest  to  be  controlled;  2)  experienced  test  pilots  have  been 
trained  to  evaluate  even  the  slightest  features  of  the  system  under  test;  and,  3) 
specially  trained  test  engineers  plan  the  tests  and  evaluate  the  resulting  data.  The 
combination  of  structured  test  and  highly  experienced  test  personnel  allows  a  detailed 
evaluation  to  occur. 

This  developmental  testing  often  detects  and  corrects  design,  or  production-related 
issues  before  the  item  goes  into  production  or  is  released  to  operational  test  or 
operational  use. 

OVERALL  FLIGHT  TESTING  CONCEPT 

While  ground  testing  of  NVSs  occurs  at  the  AFFTC,  the  primary  focus  is  on  dynamic,  in¬ 
flight  testing  scenarios.  In  some  cases,  similar  methodologies  are  used  for  ground  test 
and  flight  test.  For  example,  many  AFFTC  NVS  tests  obtain  measures  of  visual  acuity, 
and  the  same  basic  concepts  which  are  used  in  a  laboratory  setting  can  often  be  used, 
with  modification,  in  a  flight  test  setting.  Before  any  tests  are  performed,  test  pilots  who 
have  had  NVS  experience  are  selected  for  the  evaluation  team.  Qualification  currency 
with  an  existing  NVS  is  also  a  prerequisite  for  flight  tests.  In  some  programs,  pilots  are 
sent  to  a  Night  Vision  Goggle  (NVG)  refresher  training  course  at  the  onset  of  a  new  NVS 
test  program.  Prior  to  the  actual  flight  test,  the  test  pilot’s  selected  are  checked  on  the  test 
article’s  adjustment  procedures  and  are  then  subject  to  a  final  check  in  an  NVG 
compatible  eye  lane  using  standard  NVG  charts.  Their  visual  acuity  with  the  adjusted  test 
NVS  is  then  noted  and  used  as  their  test  baseline. 
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Another  step  prior  to  actual  testing  includes  proficiency  flights  with  the  system  under 
test,  without  collecting  test  point  data.  This  step  is  important  for  safety-of-flight  issues 
and  pilot  familiarity  with  the  system  under  test.  In  many  cases  routine  night  flight 
operations  are  conducted  as  part  of  this  process,  often  with  twin  seat  aircraft  with  a  pilot 
or  copilot  providing  safety-of-flight  redundancy. 

After  the  previous  system  familiarization,  and  individual  testing  has  been  completed, 
flight  test  measures  of  visual  acuity  are  initially  taken  on  the  ground  using  contrast  boards 
viewed  through  the  canopy  or  windscreen  of  an  aircraft.  Testing  always  begins  with  static 
objects  like  NVG  visual  resolution  boards,  or  ground  targets.  After  this  initial  testing  is 
completed  satisfactorily,  and  confidence  is  gained  that  the  system  will  be  safe  in  the 
flight  environment,  then  visual  assessments  with  dynamic  movement  are  introduced. 

This  testing  often  begins  with  perception  of  objects  on  the  ground,  and  includes  fixed 
targets  or  large  image  boards  which  can  be  used  to  determine  overall  visual  detection  or 
identification  ranges.  Common  final  test  points  are  associated  with  more  operational 
flight  tasks  where  the  pilot  detects  and  identifies  another  aircraft  under  dynamic  flight 
conditions.  In-flight  test  points  commonly  use  detection  or  identification  range  as  the 
primary  test  metric.  Other  variables  of  interest  always  include  measurement  of  ambient 
lighting  conditions  associated  with  the  test  flights.  Moon  phase  charts  begin  the  process 
of  assessing  the  test  lighting  conditions  but  luminance  levels  associated  with  the  actual 
test  are  double  checked  with  low-light  sensitive  photometers. 

TEST  AND  EVALUATION  METHODS 

The  AFFTC  uses  a  variety  of  methods  to  assess  NVSs.  It  is  also  common  for  the  test  and 
evaluation  of  these  systems  to  involve  a  combination  of  subjective  and  objective 
methods.  Subjective  methods  are  very  commonly  used,  and  typically  involve  techniques 
such  as  ratings,  questionnaires,  pilot  interviews,  pilot  debriefings,  and  the  generation  of 
pilot  and  engineer  concurrence  on  issues.  Objective  methods  generally  involve  data  that 
are  more  classically  described  by  objective  measures,  and  involves  the  measurement  of 
areas  like  weight,  force,  inertia,  etc. 

The  objectives  in  the  following  example  test  matrix  were  satisfied  by  test,  demonstration, 
or  analysis.  In  reference  to  AFFTC  procedures,  a  test  involves  actual  hardware  placed  in 
a  physical  test  environment,  where  tightly  controlled  variables  are  manipulated.  A 
demonstration  may  involve  the  use  of  the  same  items,  but  with  a  somewhat  less 
structured  set  of  test  points,  and  an  analysis  may  involve  the  interpretation  of  data  using 
simulation,  laboratory-type  conditions,  or  data  from  a  flight.  All  three  types  of  evaluation 
may  also  be  used  concurrently.  The  test  matrix  which  follows  is  accompanied  by  a 
discussion  of  the  specifics  of  subjective  and  objective  methods. 
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Table  1  Example  Text  Matrix  for  an  NVS 


Test  Objective 

Comments 

Type  of  Assessment 

Determine:  System  Compliance 
with  Configuration  Item 
Development  Specifications 
(CIDS),  Prime  Item  Development 
Specifications  (PIDS)  and 
Noncomplex  Item  Development 
Specifications  (NIDS) 

These  objectives  are 
specific  NVS  elements 
agreed  upon  by  the  using 
command  and  the  System 
Program  Offices  (SPOs). 
Detailed  requirements  for 
each  specific  test  program. 

Test,  Demonstration, 
and  Analysis 

Assess:  Integration  and 
compatibility  with:  hardware 
installation,  cockpit  compatibility, 
lighting,  and  aircrew  maintenance 
systems 

Generic  NVS  test  areas 

Analysis 

Assess:  Ejection  compatibility: 
ejection  seat  compatibility,  man- 
seat  separation/disconnects,  eye 
protection,  ejection  acceleration, 
windblast  resistance,  automatic 
separation,  noninterference  with 
parachute  deployment,  and 
descent  procedures 

Generic  ejection 
compatible  NVS  test 
points 

Test 

Assess:  Technical  characteristics: 
weight  and  center  of  gravity  (eg), 
structural  integrity,  and  optical 
stability 

Generic  human 
biodynamic  NVS  test 
issues 

Test,  and  Analysis 

Assess:  Human-system  interface: 
fit,  eye  protection,  comfort, 
integration  with  life  support 
equipment  and  personal  equipment, 
check-out  and  adjustment 
procedures,  donning  and  doffing, 
ingress  and  egress,  and  maintainer 
interface,  physiology  including 
Valsalva  procedure 

Core  Human  Factors 

NVS  integration  areas 

Demonstration  and 
Analysis 

Assess:  Non-combat  missions, 
special  operations,  ground  ops 
support,  goggle  performance, 
diagnostics,  and  built-in  test 
capability. 

Determines  special  uses 
utility 

Demonstration 

Assess:  Reliability  and 
maintainability  elements 

Determines  mechanical 
and  system  support, 
durability  and 
maintainability  over  time 

Analysis 
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Subjective  Test  Methods 


All  test  methods  used  at  the  AFFTC  ultimately  lead  to  a  final  subsystem  or  system 
conclusion  in  three  summary  categories:  Satisfactory,  Marginal,  or  Unsatisfactory.  These 
categories  reflect  all  test  elements,  and  provide  a  final  judgment  involving  all  elements 
and  requirements  being  tested. 

Many  specific  subjective  approaches  have  been  used  at  the  AFFTC  to  contribute  to  these 
three  final  conclusion  possibilities.  Typically,  the  beginning  stages  of  the  AFFTC  test 
approach  reflect  on  the  ability  to  perform  the  mission  in  terms  of  specific  requirements 
set  by  the  user  command.  These  requirements  are  usually  translated  by  the  SPO  and  the 
AFFTC  into  test  points,  which  are,  in  turn,  assessed  by  subjective  and  objective 
measures.  Occasionally,  the  contractor  of  the  system  under  test  is  involved  with  both  test 
planning  and  test  conduct,  usually  under  an  integrated  product  team  (IPT)  concept  with 
the  AFFTC  and  SPO. 

Some  test  programs  have  used  open-ended  subjective  inquiry,  some  have  used  yes  or  no 
questions  and  some  have  used  rating  scale  approaches.  Overall,  the  rating  scale  approach 
is  the  most  popular,  followed  by  yes/no  types  of  questions  and  then  finally  by  an  open- 
ended  subjective  inquiry.  Each  approach  has  advantages  and  disadvantages,  and  is 
tailored  to  specific  systems  evaluations.  For  example,  an  open-ended  questionnaire  may 
be  used  on  an  entirely  new  system  when  the  capabilities  of  the  test  article  are  not  well 
defined,  or  when  the  system  has  never  been  flown  in  a  particular  aircraft.  The  first  NVSs 
programs  at  the  AFFTC  used  such  an  approach  when  NVSs  were  new  technology  in 
aircraft.  Later  programs  became  more  comparative  with  comparisons  being  made 
between  earlier  NVSs  and  later  NVSs  or  system  enhancements.  A  primary  benefit  to 
subjective  testing  is  that  the  pilot’s  previous  experience  serves  as  a  benchmark  to 
measure  against. 

Numeric  values  created  by  various  subjective  rating  systems  assess  positive  and  negative 
elements  of  the  systems  under  test  and  then  may  be  aggregated  into  conclusions  by 
subsystem  and  then  by  the  system  overall.  This  process  can  include  considerable  reliance 
on  specialized  descriptive  and  inferential  statistics  which  translate  numerous  subjective 
rating  values  into  summary  conclusions.  Being  a  subjective  process  overall  there  is  also  a 
reliance  on  engineering  and  aircrew  judgment  to  qualify  the  impact  of  numeric  ratings, 
and  to  further  refine  the  overall  importance  of  the  results  obtained. 

The  AFFTC’s  Human  System  Integration  Branch  in  particular,  has  expended 
considerable  effort  to  ensure  that  subjective  rating  systems  are  industry  accepted, 
systematic,  psychometrically  sound,  linguistically  balanced,  and  not  subject  to  excessive 
response  bias.  Historic,  subjective  rating  systems  such  as  the  Cooper-Harper,  Bedford, 
United  States  Air  Force  School  of  Aerospace  Medicine  (USAFSAM)  and  AFFTC 
six-point  rating  scale  are  often  supplemented  by  more  specialized  subjective  tools  such  as 
the  Subjective  Workload  Assessment  Technique  (SWAT).  It  is  worthy  to  note  that  there 
is  no  one  method  used  overall  at  the  AFFTC  to  assess  NVS  subjective  data.  Subjective 
methods  are  chosen  by  team  efforts  on  the  part  of  the  AFFTC,  the  SPOs,  the  item 
contractor,  and  the  customer.  The  only  overall  AFFTC  direction  provided  for  systems 
under  test  is  that  the  final  judgment  reaches  only  one  of  the  following  conclusions: 
satisfactory,  marginal  or  unsatisfactory. 
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Objective  Test  Methods 


The  use  of  objective  methods  for  NVSs  often  relies  on  dedicated  test  points  accomplished 
on  the  ground,  or  during  in-flight  tests.  Objective  data  also  involves  measurement  of 
visual  processes  associated  with  NVS. 

Because  NVSs  are  currently  a  human-mounted  system,  the  objective  data  accruing  from 
tests  are  often  intermingled  with  subjective  judgments.  As  another  example,  biodynamic 
measures  of  helmet  lift  loads,  or  center  of  gravity  for  an  NVG  system  can  result  in 
calculations  of  head  and  neck  loads  which  are  imparted  to  an  aircrew  during  dynamic 
flight.  While  these  measures  can  be  calculated  using  an  objective  engineering  approach, 
their  impact  on  aircrew  can  still  result  in  a  high  variability  of  subjective  response.  A 
certain  neck  load  for  a  small  pilot  (5th  percent  in  body  weight)  may  be  excessive  and 
result  in  unacceptable  strain,  whereas  during  the  same  flight  conditions,  using  the  same 
equipment,  the  test  may  result  in  no  discomfort  for  a  larger  pilot  (95th  percent  body 
weight).  In  some  cases  there  may  be  an  objective  criterion  such  as  a  maximum  neck  load 
(representing  a  worst  case  situation)  that  would  result  in  a  pass/fail  assessment,  thereby 
precluding  a  subjective  judgment  of  discomfort.  In  this  case,  and  in  many  cases  of 
human-machine  interaction,  the  objective  data  imparted  by  a  loads  analysis  must  be 
tempered  with  both  the  subject’s  physical  characteristics  as  well  as  the  subjective  effect 
that  system  loading  has  on  the  human’s  performance  and  well  being. 

One  of  the  objective  measures  collected  for  such  tests  includes  measures  of  anthropometry 
on  the  test  subjects  selected.  Anthropometries  involves  the  measurement  of  human  body 
dimensions,  with  the  goal  of  comparing  individuals  to  a  larger  population.  Using  this 
objective  tool,  inferences  can  be  made  about  the  suitability  of  an  NVS  for  the  larger 
population  of  USAF  aircrew.  Using  the  test  subject  as  their  own  control,  subjective 
response  data  can  also  be  coupled  with  objective  comparisons  of  that  same  subject. 

Other  types  of  objective  data  tested  are  derived  from  NVS  program  specifications,  and 
include  areas  such  as  windblast  resistance,  weight,  and  center-of-gravity  issues. 

Examples  of  these  tests  would  include  observation  and  analysis  of  the  physical  properties 
of  the  NVS  under  specific  test  conditions,  for  example,  system  response  to  a  400-knot 
windblast.  Other  tests  would  include  observation  and  analysis  of  goggle  trajectories 
during  sled  test  ejections  of  mannequins  wearing  ejection  compatible  goggles.  These 
types  of  tests  usually  involve  visual  analysis  of  high-speed  video,  and  subsequent  visual 
inspection  of  the  test  article. 

Examples  of  AFFTC  NVS  Test  Methods  for  Various  Programs 

YA-10B  Aircraft  and  AN/AVS-6  Night  Vision  Goggles 

One  of  the  earliest  NVS  tests  to  occur  at  the  AFFTC  was  the  testing  of  the  AN/AVS-6 
NVG.  Accomplished  from  October  1982  through  January  1983,  this  test  program  used 
the  YA-10B  aircraft  as  a  flying  test  bed,  with  additional  ground  tests  performed  on  the 
F-15B,  F-16A,  F-l  1  ID  and  A-7D.  Objectives  for  the  test  involved  assessing  the 
usability  of  those  goggles  for  night  attack  roles,  formation  flying,  low  level  navigation, 
air-to-ground  gunnery,  as  well  as  integration  with  limited  and  full  avionics  suites. 
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Additional  objectives  included  determining  minimum  ambient  light  levels  for  navigation 
using  the  NVGs  as  sole  sensor,  determining  avionics  display  interactions,  such  as  the  use 
of  forward  looking  infrared  radar  (FLIR),  the  employment  of  illumination  flares,  the 
effects  of  cockpit  lighting  modifications  and  the  overall  impact  on  pilot  workload.  The 
in-flight  tests  were  primarily  conducted  at  low  level  (200  to  300  feet  Above  Ground 
Level  [AGL])  over  open  terrain,  and  at  500  feet  AGL  in  mountainous  terrain. 

Specific  methods  used  in  this  test  included  assessments  of  in-flight  conspicuity  of  chase 
or  lead  aircraft,  and  the  ability  to  take-off  and  land  with  runway  lights  blacked  out. 
Lighting  conditions  were  manipulated  from  clear  starlight  to  half  moon  or  greater 
illumination.  Interactions  with  head-up  display  (HUD)  and  FLIR  imagery  were  assessed 
as  well  as  visual  perception  issues  such  as  determination  of  distance  to  clouds,  and 
discrimination  of  terrain  features  such  as  hills,  mountains,  deserts  and  bodies  of  water. 
Pilot  workload  was  assessed  using  the  SWAT  subjective  workload  rating  system.  A 
list  of  open-ended  questions  were  asked  of  the  pilots  to  assess  visual  issues  such  as 
field-of-view,  field-of-regard,  and  depth  perception  for  10  separate  flight  operations. 
Additional  questions  covered  avionics  compatibility  with  HUD  and  raster  video,  canopy 
lighting,  reflections,  dark  adaptation  time,  goggle  adjustment,  gun  flash  effects  on  vision, 
maneuverability,  detection  ranges,  g-effects,  and  effects  on  instrument  crosscheck.  In 
addition,  several  general  questions  were  asked  relating  to  the  necessity  and  utility  of  the 
NVGs  for  their  intended  mission,  as  well  as  questions  discussing  strong  points  and 
weaknesses  of  the  goggles.  Since  this  was  the  first  major  in-flight  test  conducted  at  the 
AFFTC,  many  open-ended  questions  were  asked  in  order  to  determine  an  overall 
understanding  of  how  the  AN/AVS-6  would  interact  with  a  fighter  aircraft  under 
numerous  test  conditions. 

MC-130H  Combat  Talon  II  Aircraft 

From  1987  to  1989,  a  test  program  was  conducted  on  the  MC-130H  Combat  Talon 
aircraft.  The  purpose  of  that  test  was  to  assess  the  compatibility  of  the  aircraft  lighting 
with  NVG  systems 

F-15  and  F-16  Testing  of  Cats  Eyes  and  Nite  Ops  Goggles 

In  1991  a  small  series  of  tests  were  completed  on  the  F-15  and  F-16  aircraft  of  the 
developmental  Cats  Eyes  and  Nite  Ops  NVG  systems.  A  seven-point  rating  scale  of 
acceptability  was  used  for  the  ground  evaluations.  Areas  assessed  subjectively  included: 
donning  and  doffing,  interpupillary  goggle  adjustment,  eyepiece  focus,  vertical 
adjustment,  fore  and  aft  as  well  as  tilt  adjustment,  eye  relief,  peripheral  vision  and  display 
and  cockpit  lighting.  Testing  included  a  variety  of  static  cockpit  compatibility  tests, 
followed  by  limited  flight  testing.  A  separate  F-15  night  tactics  test  questionnaire  was 
also  used  for  conventional  USAF  NVGs  (F4949)  in  the  same  time  frame. 

C-17A  Aircraft 

From  July  1992  to  March  1995  an  assessment  was  made  of  the  C-17A’s  compatibility 
with  the  projected  image,  Type  II,  Class  B,  GEC  Mark  IV  Cats  Eyes,  as  well  as  the 
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International  Telephone  and  Telegraph  (ITT)  F4949  Type  I,  (with  Class  A  and  Class  B 
filters)  NVG.  Three  areas  of  interest  for  this  test  were:  ground  testing,  flight  testing 
above  10,000  feet  AGL,  and  operationally  representative  low-level  flight  missions  (at  or 
above  900  feet  AGL).  A  six-point,  bipolar,  subjective  scale  of  adequacy  was  used. 

Areas  of  interest  included  primary  flight  displays  and  controls,  external  visibility, 
windshield  light  transmittance,  and  external  night  visual  acuity.  Flight  segments 
included:  preflight,  cruise  at  12,500  feet,  a  3-degree  approach,  a  go-around,  a  5-degree 
approach,  and  an  airdrop.  Moon  conditions  were  48  percent  illumination  for  the  in-flight 
tests.  Ground  tests  used  a  projected  image  bar  chart  display.  Luminance  was  controlled 
for  ground  tests  by  placing  the  aircraft  in  the  USAF  Benefield  Anechoic  Facility  (BAF) 
where  lighting  levels  could  be  precisely  controlled.  For  certain  tests  where  exact 
specification  lighting  levels  were  specified,  but  could  not  be  met  exactly,  the  differences 
in  lighting  from  those  conditions  were  compensated  for  by  use  of  a  scaling  factor  method. 

F-16  NVG  Testing 

Numerous  NVSs  have  been  tested  on  the  F-16  aircraft.  Testing  on  Block  25/30  and  32 
aircraft  in  1996  covered  the  following:  egress,  maintainability,  transportability,  glare, 
reflections,  light  leaks,  cockpit  lighting,  and  perception  of  labels/legends/displays. 
Assessments  were  made  of  minimum  resolvable  angles  through  line  grading  charts  or  the 
use  of  a  Bailey-Lovie  visual  acuity  chart.  Ground  testing  also  involved  the  external 
perception  of  the  aircraft  at  0,  45,  90,  135  and  180  degrees  aspect  angle,  with  aircraft 
detection,  identification  and  aspect  angle  being  test  criteria.  Flight  tests  involved 
subjective  ratings  using  a  six-point  rating  scale  and  were  made  for  formation,  low-level 
navigation,  air-to-ground,  air-to-air  attack,  and  instrument  flight.  Later  testing  of  the 
F-16  C/D  involved  special  operations  flight  profiles. 

AN/AVS-8  (V)  Night  Vision  System 

This  NVS  was  flight  tested  between  1996  and  1997  and  consisted  of  three  versions. 
Versions  1  and  2  were  ejection  compatible  goggles  containing  new  eye  protection  visors, 
and  an  integrated  chin  and  nape  strap  for  the  base  helmet.  Version  3  was  a  helicopter 
version  with  a  sand  and  dust  visor.  Testing  of  this  new  goggle  system  was  extensive  and 
involved  lighting,  windblast,  ejection  tower,  goggle-helmet  separation,  man-seat 
separation,  centrifuge  testing,  weight  and  eg  testing,  hanging  harness,  sled  tests,  and 
reliability  and  maintainability  issues.  Laboratory  testing  was  completed  in  association  with 
flight  testing  and  the  AFFTC  was  the  responsible  test  organization  (RTO).  Additional 
laboratory  testing  was  conducted  at  a  variety  of  specialized  NVS  testing  laboratories. 

This  NVS  was  tested  by  the  AFFTC  in  an  F-16,  C-141  and  C-17  from  overcast  starlight 
levels  to  as  much  as  .89  moon  illumination.  This  test  program  used  standard  F4949G 
NVGs  as  control  goggles  for  the  test. 

Windblast  testing  involved  speeds  from  350  to  600  knots  in  A-10  and  B-1B  simulated 
cockpits.  During  live  windblast  tests  an  instrumented  Hybrid  III  95th  percentile  male 
mannequin  was  used.  The  mannequin  was  outfitted  with  combinations  of  standard  life 
support  gear  including:  HGU-55/P  helmet,  MBU-12/P  and  MBU-20/P  masks,  the  LPU-9 
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support  gear  including:  HGU-55/P  helmet,  MBU-12/P  and  MBU-20/P  masks,  the  LPU-9 
life  preserver  and  SRU-21/P,  SRU-12/P,  and  CSU-17/P  survival  vests,  the  PCU-15/P 
torso  harness  with  CRU-60  or  CRU-94  connectors,  the  CWU-27/P  flight  suit,  and  the 
CSU-13B/P  anti-g  suit.  The  mannequin  was  instrumented  with  a  Denton  load  cell  to 
measure  neck  lift  loads  and  moments.  Numerous  tests  were  accomplished  using  several 
configurations  of  the  ACES  II  ejection  seat  system. 

Ejection  tower  tests  were  accomplished  using  A-10,  B-1B,  and  F-16  configured  seat  systems. 
These  tests  assessed  general  ejection  compatibility  prior  to  more  extreme  rocket  sled  tests. 
Ejection  tower  tests  used  5th  and  95th  percentile  Aerospace  Hybrid  II  mannequins. 

Centrifuge  testing  included  assessments  of  helmet  and  goggle  movement  with  a  live  test 
subject.  Instrumentation  determined  the  degree  of  movement  of  the  NVS  as  well  as  the 
pilot’s  helmet. 

Sled  tests  used  an  F-16C,  F-16D  and  A-10A  fore-body  with  an  ACES  II  configured  seat. 
Like  the  ejection  tower  tests,  the  sled  tests  used  Hybrid  III  male  mannequins.  Sled  test 
speeds  varied  from  approximately  250  to  over  600  knots.  Sled  testing  is  significantly 
more  dynamic  than  tower  testing  since  the  dual  elements  of  full  ejection  loading  is 
coupled  with  high-speed  windblast  as  the  mannequin  ejects,  and  is  recovered  by  the  a 
parachute  process.  Flight  testing  occurred  on  Air  Mobility  Command  (AMC)  aircraft 
using  operational  aircrew,  rather  than  AFFTC  test  pilots.  It  included  testing  of  the  second 
version  NVS  on  C-5,  C-130,  C-141,  KC-135,  and  C-17  aircraft.  A  simple  six-point, 
bipolar  1  to  6  numeric  value  rating  scale  using  descriptive  adjectives  measuring  levels  of 
satisfaction  was  used.  This  test  also  used  a  1  to  5  numeric  and  descriptive  adjective 
rating  scale  to  compare  the  NVS  second  version  to  both  the  earlier  production  ANVIS-6 
as  well  as  production  F4949  NVGs. 

The  subjective  measures  of  effectiveness  used  on  the  flight  portion  of  the  test  included 
assessments  of  whether  the  NVS  version  2  was  meeting  or  exceeding  the  capability  of 
the  current  NVGs  (F4949  NVGs)  then  in  use  by  that  USAF  command.  Test  areas 
included:  meeting  mission  tasks,  meeting  mechanical  and  physical  requirements  of  flight, 
ability  to  see  inside  the  cockpit,  to  acquire  external  targets,  meet  all  aviation 
requirements,  training  requirements,  maintenance  instructions  and  documentation, 
mobility  and  transportation  requirements,  egress/escape  requirements,  and  to  assess  its 
reliability  to  support  mission  requirements. 

Specific  Human  Factors  form-fit-function  questionnaires  were  also  used  in  this  test,  in  an 
F-16  aircraft  on  the  ground  and  pilot  ratings  on  a  1  to  6  numeric  rating  scale  of 
satisfaction  were  used.  This  bipolar  scale  had  no  neutral  point  and  had  three  categories  of 
satisfactory  and  three  levels  of  unsatisfactory  for  the  scale.  This  subjective  measure 
rated:  goggle-to-aircraft  connection  and  disconnection  capability,  one  hand  and  two  hand 
goggle  manipulation,  donning  and  doffing  procedures,  eye  protection  with  and  without 
spectacles,  ability  to  perform  the  Valsalva  procedure,  head  mobility,  comfort,  helmet 
balance,  as  well  as  including  an  overall  numeric  rating  for  the  system. 

Finally,  the  system  was  tested  for  maintainability  and  reliability  factors  which  included: 
module  replacement,  time  to  repair,  adjustment  capability,  independence  of  failures, 
maintainer  skill  levels  required  to  repair,  organizational  level  repair,  preventative 
maintenance  ease,  and  fault  isolation. 
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C-130J  Aircraft 


This  evaluation  was  conducted  from  April  1998  through  June  1999  and  involved  both 
ground  and  in-flight  testing  of  F4949L  (Class  B)  and  F4949D  (Class  A)  NVGs  in  the 
C-130J.  Areas  of  interest  were  cockpit  compatibility  as  well  as  the  cargo  area.  In  the 
cargo  compartment,  overhead  lighting,  floor  lighting,  jump  lighting,  and  aft  side  door 
lighting  were  assessed.  Instrumentation  lighting  was  tested  at  specified  luminance  levels 
of  approximately  .1  foot-Lambert  (fL)  for  primary  and  secondary  displays,  and 
multicolor  and  monochrome  displays  were  specified  at  .5  fL  ambient.  The  windscreen 
and  HUD  combiner  were  also  areas  of  interest. 

C-141C  Aircraft 

Night  vision  systems  tested  in  the  C-141C  in  1999  included  F4949D  and  F4949G  goggles 
with  emphasis  placed  on  cockpit  display  NVS  compatibility,  as  well  as  external  visibility 
visual  acuity  tests.  Ground  testing  of  external  visibility  at  night  involved  the  use  of  the 
Medium  Contrast  Resolution  Resolving  Power  Target  (USAF  Tri-Bar  chart).  Visual 
acuity  was  based  on  the  conversion  of  the  USAF  tri-bar  charts  to  Snellen  visual  acuity 
values.  Visual  acuity  values  were  analyzed  and  then  extrapolated  to  determine  visual 
acuity  potential  in  flight. 

F-22  and  F/A-22  Aircraft 

The  AFFTC  assessed  the  F-22  aircraft  using  conventional  F4949  NVGs  by  laboratory 
and  simulator  analysis  from  1997  to  2000,  and  on  the  F/A-22  from  2000  to  the  date  of 
this  report.  Cockpit  compatibility  tests  were  performed  to  ensure  that  the  F4949  could  be 
used  in  the  F-22,  and  that  its  cockpit  and  external  lighting  was  NVG  compatible  lighting. 
Test  areas  of  interest  included:  pilot  life  support  equipment  physical  compatibility, 
overall  cockpit  fit  compatibility,  specific  cockpit  night  lighting  capability,  external 
aircraft  lighting  NVG  compatibility,  and  external  pilot  vision  capability.  Ground  tests 
were  performed  in  special  hangars  with  natural  moon  light  levels  providing 
approximately  50  percent  moon  luminance  equivalent.  Psychometric  visual  perception 
charts  were  developed  to  measure  test  light  levels.  Those  charts  were  based  on  spectral 
color  perception  and  object  visual  resolution  capabilities  of  a  USAF  pilot  population. 
These  charts  used  colors  and  objects  of  known  sizes  to  correlate  known  resolution  and 
visual  perception  of  the  USAF  population  to  the  specifics  of  a  particular  test  group,  in 
effect,  establishing  what  the  low  light  levels  were  without  relying  on  a  photometer. 

Objectives  for  the  F/A-22  for  NVS  included  fully  compatible  internal  and  external 
lighting,  pilot  external  visual  acuity  with  NVGs,  and  life  support  equipment  compatibility 
with  F4949  NVGs.  Objective  data  collected  included  measurement  of  light  emissions  for 
NVG  compatible  lighting,  and  interaction  with  displays  of  the  aircraft.  Pilot  subjective 
comments  on  the  suitability  of  the  NVGs  with  the  aircraft  were  also  collected  and 
analyzed.  Other  areas  of  interest  were  assessments  of  reflection  ejection  suitability, 
windblast,  stowage,  donning  and  doffing,  and  pilot  ingress/egress.  The  AFFTC  is 
continuing  to  assess  NVS  compatibility  with  the  F/A-22  aircraft. 
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Integrated  Panoramic  Night  Vision  Goggles  (IPNVGs) 

This  test  program  was  conducted  between  March  and  April  of  2003  and  involved  the 
testing  and  comparison  of  the  Integrated  Panoramic  Night  Vision  Goggle  (IPNVG) 
with  USAF  issue  AN/AVS-9  (F4949)  NVGs.  The  test  aircraft  was  an  F-16B,  and  the 
test  was  conducted  through  the  USAF  Test  Pilot  School.  The  IPNVG  provides  a  wider 
field-of-  view  than  issue  NVGs,  and  is  worn  on  a  conventional  flight  helmet  the  same  as 
USAF  issue  NVGs. 

Objectives  included  comparing  situation  awareness,  workload,  and  effectiveness  of 
IPNVGs  to  F4949  NVGs.  Luminance  levels  for  the  test  were  in  two  categories:  0  to 
25  percent  moon  illumination  and  75  tolOO  percent  moon  illumination. 

This  test  used  pilot  physical  performance  time  for  a  matched  task,  within-subjects 
assessment  of  the  effect  each  NVG  had  on  pilot  situation  awareness.  This  was  a  direct 
assessment  of  situation  awareness  as  measured  by  an  individual’s  performance  times  on 
several  tasks  where  two  different  NVGs  were  worn.  The  same  flight  profiles  involving 
unusual  attitude  recovery,  radar  cursor  movement  reaction  time,  and  target  visual  capture 
time  were  also  used  for  assessing  workload  using  the  two  types  of  NVGs.  Pilot  attention 
to  a  radar  cursor  position  change  while  performing  a  station  keeping  flight  profile 
assessed  the  pilot’s  level  of  situation  awareness.  In  another  test  case,  the  time  required  to 
acquire  and  capture  a  target  on  the  HUD  was  used  to  assess  situation  awareness  of  the 
target  as  being  affected  by  NVG  type. 

The  workload  assessment  was  based  on  comments,  and  subjective  ratings  provided 
through  the  AFFTC  seven-point  workload  estimate  scale.  This  workload  scale  rates 
workload  on  a  range  from  1  (nothing  to  do)  to  7  (overloaded,  system  unmanageable).  Its 
ordinal-level  resultant  values  were  compared  and  analyzed  using  non-parametric 
statistical  tests. 

The  suitability  of  the  IPNVGs  assessed  using  several  flight  profiles  also  included 
assessments  of  field-of-view  when  maneuvering  under  high  g-loads,  and  an  assessment  of 
the  military  utility  of  the  IPNVG  system. 

CV-22  Tilt-rotor  Air  Vehicle 

The  new  tilt-rotor  CV-22  air  vehicle  will  be  assessing  the  use  of  conventional  NVSs  in 
several  areas:  cockpit  compatibility,  gauge  visibility,  display  compatibility,  HUD 
compatibility,  and  functionality  of  HUD  with  NVS.  These  test  points  involve  a  variety  of 
flight  conditions,  as  well  as  use  by  left  or  right  seat  aviators.  Assessment  ratings  will  use 
the  AFFTC  six-point  rating  scale  for  absolute  ratings  of  operator  satisfaction.  Flight 
testing  is  beginning  on  this  new  air  vehicle  (2003)  with  NVS  integration  testing  currently 
being  scheduled  in  test  plans. 
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TEST  METHODOLOGY  DISCUSSION 


Test  Item  Evaluation  Criterion 

The  establishment  of  test  item  criterion  is  usually  governed  by  a  concurrence  of  the  AFFTC 
testers,  a  SPO,  and  the  using  Command.  Program  objectives  are  often  written 
of  requirement  thresholds  and  objectives,  and  from  those,  the  system  objectives  are 
often  used  as  a  base  criterion  for  flight  testing.  In  the  case  of  objective  data,  pass/fail 
criterion  are  relatively  easy  to  set.  For  example,  if  an  item  is  designed  to  survive  a 
600-knot  windblast  test  and  remain  fully  functional,  the  subsequent  inspection  of  that  item 
post-windblast  can  pass  or  fail  that  requirement  based  on  a  single  test.  However,  single¬ 
event  tests  are  not  always  the  accepted  test  approach,  and  the  ability  to  pass  a  system  often 
hinges  on  several  trials  under  differing  test  conditions.  Providing  multiple  test  events  for 
any  particular  requirement  builds  confidence  in  the  ability  of  that  product  to  successfully 
complete  its  role  in  an  operational  setting,  as  well  as  building  a  reliability  history,  which 
also  instills  confidence  in  the  test  approach  and  sustained  durability  of  that  product. 

Build-up  and  Additional  Testing 

Requirements  are  often  tested  using  a  build-up  approach,  where  progressively  more 
severe  limits  are  reached  and  then  exceeded.  The  build-up  approach  provides  a  safe 
approach  to  determining  product  limits,  while  providing  good  data  on  all  the  ranges  of 
the  systems  operational  envelope. 

Variation  in  the  pass/fail  approach  for  objective  requirements  varies  by  program,  with 
smaller  programs  typically  limiting  test  redundancy  for  budget  or  other  concerns,  while 
larger  programs,  or  more  complex  weapon  systems,  warrant  more  repetition  during  test. 

In  all  cases,  time  and  schedule  drive  the  ability  to  collect  data,  and  thus  provide 
constraints  under  which  pass/fail  conclusions  can  be  drawn.  However;  whenever 
possible,  more  testing  results  in  greater  confidence  in  the  final  conclusions. 

SUMMARY 

Night  vision  systems  have  been  and  are  being  tested  on  a  regular  basis  at  the  United 
States  AFFTC.  Performing  developmental  test  on  NVSs  allows  them  to  either  be 
immediately  fielded,  or  moved  on  to  operational  testing  by  an  independent  operational 
testing  agency  (OTA).  For  the  USAF  that  OTA  is  AFOTEC. 

Central  to  the  AFFTC  testing  concept  for  NVS  is  the  idea  that  the  AFFTC  can  often 
provide  the  first  integrated  developmental  tests,  in  a  variety  of  airframes  and  under  tightly 
controlled  conditions.  Our  ability  to  provide  dynamic  flight  environments  with  highly 
trained  test  pilots  and  correspondingly  trained  flight  test  engineers  provides  a  typical 
first-look  at  the  possible  operational  utility  of  new  or  modified  system. 

Methods  used  at  the  AFFTC  include  both  objective  and  subjective  data  collection  and 
analysis.  Testing  often  follows  a  build-up  approach  and  then  an  envelope  expansion  stage 
where  the  items  are  tested  under  dynamic  conditions  to  their  design  limits  or  beyond. 
Regression  testing  provides  redundant  test  efforts  in  order  to  build  confidence  in  the 
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projected  utility  the  item  will  have  during  its  operational  life.  Analysis  of  all  data 
collected  during  the  evaluation  is  complex,  but  it  is  particularly  complex  when  dealing 
with  subjectively  derived  performance  requirements  or  specifications.  However, 
specially  trained  engineers,  test  pilots,  and  integrated  product  teams  can  provide  such 
analysis  and  insight  to  ensure  that  the  developmental  product  received  for  test  can  be 
confidently  released  for  field  use  or  further  operational  testing. 

Acronyms 

AFB  -  Air  Force  Base 

AFFTC  -  Air  Force  Flight  Test  Center 

AFOTEC  -  Air  Force  Operational  Test  and  Evaluation  Center 

AGL  -  above  ground  level 

AMC  -  Air  Mobility  Command 

BAF  -  Benefield  Anechoic  Facility 

CA  -  California 

eg  -  center  of  gravity 

CIDS  -  Configuration  Item  Development  Specifications 
fL  -  foot-Lambert 

IPNVG  -  Integrated  Panoramic  Night  Vision  Goggle 

IPT  -  integrated  product  team 

ITT  -  International  Telephone  and  Telegraph 

NIDS  -  Noncomplex  Item  Development  Specifications 

NVG  -  Night  vision  goggles 

NVS  -  Night  vision  systems 

PIDS  -  Prime  Item  Development  Specifications 

RTO  -  responsible  test  organization 

SPO  System  Program  Office 

SWAT  -  Subjective  Workload  Assessment  Technique 
USAF  -  United  States  Air  Force 

USAFSAM  -  Untied  States  Air  Force  School  of  Aerospace  Medicine 
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